Frame syncrhonization in a dual-aperture camera system

ABSTRACT

A dual-aperture camera comprising a first camera having a first sensor and a first image signal processor (ISP), the first camera operative to output a first stream of frames, a second camera having a second sensor and a second ISP, the second camera operative to output a second stream of frames, and a synchronization and operation control module configurable to control operation of one camera in a fully operational mode and operation of the other camera in a partially operational mode and to output an output of the fully operational camera as a dual-aperture camera output, whereby the partially operational mode of the other camera reduces a dual-aperture camera the power consumption in comparison with a full operational mode of the other camera.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a Continuation from U.S. patent application Ser. No.16/055,120 filed Aug. 5, 2018 (now allowed), which was a Continuationfrom U.S. patent application Ser. No. 15/570,346 filed Oct. 29, 2017(issued as U.S. Pat. No. 10,616,484), which was a 371 application frominternational patent application PCT/IB2017/053470, and is related toand claims priority from U.S. Provisional Patent Application No.62/351,990 filed Jun. 19, 2016, which is incorporated herein byreference in its entirety.

FIELD

Embodiments disclosed herein relate in general to camera systems thatcomprise two or more cameras (also referred to as “dual-cameras” or“dual-aperture cameras”) and are connected to host devices that processcamera data and require synchronization between frames output by two ormore cameras to reduce power consumption.

BACKGROUND

Digital camera modules are currently being incorporated into a varietyof host devices. Such host devices include cellular telephones, personaldata assistants (PDAs), computers, and so forth. Consumer demand fordigital camera modules in host devices continues to grow.

There is an ever-increasing demand from host device manufacturers toinclude higher-performance cameras, with better capabilities such asoptical zoom, improved low-light performance and higher image quality.To tackle this demand, new camera systems have been proposed recently.Such camera systems include two cameras aligned to look in the samedirection, with partially or fully overlapping fields of view (FOVs) andare referred to herein as “dual-camera” systems (or “dual-aperturecamera” systems, with two apertures A and B), see e.g. internationalpatent applications PCT/IB2014/062180, PCT/IB2014/063393 andPCT/IB2016/050844. The two cameras may have similar FOVs or verydifferent FOVs, depending on the lenses used in each. It has been shown(see e.g. PCT/IB2014/062180 and PCT/IB2014/063393) that the images fromthe two cameras may be “stitched” or “fused” together according to adedicated algorithm to form a composite image, with improved resolution,improved noise performance and improved image quality (at least for somepart of the composite image field of view). The image stitching or imagefusion algorithm can be implemented in software, running on anapplication processor (AP), or in hardware (hard-wired implementation).

It has also been shown (see e.g. co-owned U.S. Pat. No. 9,185,291) thatsome dual-camera systems, such as ones that provide high-quality zoomduring preview or video recording or as ones that provide enhanced lowlight performance, may include a transition between one camera stream tothe other camera stream in order to generate an output stream of frames,which is used in turn to show the preview or to record the video. Thistransition takes place at a certain zoom factor (ZF) when zooming in andout. In some cases, it is beneficial to keep the transition between thetwo cameras as smooth as possible—for example, in case the two camerasin the dual-camera system have different FOVs and where the dual-camerasystem implements continuous zooming between the two cameras. A smoothtransition is a transition in which the user does not notice thetransition point between the two cameras. A smooth transition should besmooth in time and space, namely continuous in both aspects.

Furthermore, it is known that some dual-camera systems may includecalculation of a depth map from the two camera frames. A depth map is amap of the pixels in the frame, in which each object's relative distancein a scene is determined from the spatial shift of the object's imagebetween the two frames. In some embodiments, the depth map requires aregistration step between the frames from the two cameras. Aregistration step is a step in which a match is found between pixels inthe two images that correspond to the same object in the scene, and adisparity value that represents the offset between the location on thesensor of the two corresponding pixels is assigned to each pair ofmatched pixels to form a “dense disparity map”. Alternatively, aregistration step may include extracting features from the two frames,finding matches between features corresponding to the same object in thescene and calculating from the matched features a “sparse depth map”.The depth map may be calculated on a preview or video stream, or on asnapshot image.

For the three applications mentioned above (fusion of two capturedimages, transition between two streams of frames and creating a depthmap from two camera frames), the synchronization of the acquisition timeof the frames is an important requirement and common practice. Forexample, when registering information between two frames from the twocameras, any object motion in the scene or motion of the dual-aperturecamera may result in registration errors if the frame acquisition timeis not synchronized within a certain period of time (e.g. less than 3-5msec). The registration errors can lead to wrong depth estimations whencalculating a depth map. In smooth transition, lack of synchronizationin time between pairs of frames from the two cameras may lead to anoticeable discontinuity when switching from one camera to the other.

A known in the art synchronization method between two camera sensorsincludes sending a synchronization signal every frame from one sensor,denoted “master sensor”, to the second sensor, denoted “slave sensor”.This method requires the two cameras to output the stream of frames atapproximately the same rate to stay synchronized (for example, bothsensors will output the frames at a rate of 30 fps).

Apart from maintaining synchronization, there are other benefits tokeeping the two cameras streaming in parallel at all times (even whenonly one camera is actually used to generate the output image or frame):first, it is desired to maintain accurate information of focus, whitebalance and light gain level (known as “3A information”) for bothcameras, even when one is not used, in order to be able to useinformation from the not used camera with as small a latency aspossible. If one camera is set to be in “standby” mode and does notstream frames, it may take up to several seconds until white balance,exposure and focus converge to values that match the scene whenconfiguring the camera to start streaming frames. This time may hinderuser experience and may prevent smooth transition from one camera to theother, for example when zooming-in or zooming-out, or for example whenswitching from regular light mode to low light mode. Second,registration may be required to be maintained at all times, for examplefor the purpose of calculating a depth map of the scene from the twoimages. However, running two camera sensors in parallel carries thepenalty of doubling power consumption.

In summary, to enable fast output switching between one aperture(camera) and another aperture (camera) in a dual-camera, both cameraneed to be operative and synchronized. This creates a power consumptionproblem, since keeping two cameras fully operational results in doublingthe combined camera power consumption in comparison with that of asingle camera system. At present, there are no satisfactory solutions tothis power consumption problem.

SUMMARY

In exemplary embodiments, there is provided a system comprising adual-aperture camera that includes a first camera operative to output arespective first camera output and a second camera operative to output arespective second camera output, and a synchronization and operationcontrol module configurable to control operation of one camera in afully operational mode and operation of the other camera in a partiallyoperational mode, whereby operation of the dual-aperture camera with onecamera in partially operational mode and another camera in fullyoperational mode reduces system power consumption in comparison with thesystem power consumption when both cameras operate in fully operationalmode.

In an exemplary embodiment, the synchronization and operation controlmodule is further configurable to output the output of the fullyoperational camera as a dual-aperture camera output.

In an exemplary embodiment, the first camera includes a first cameraimage sensor that communicates with an associated first image signalprocessor (ISP) and is operative to output a first stream of frames, thesecond camera includes a second camera image sensor that communicateswith an associated second ISP and is operative to output a second streamof frames, and the synchronization and operation control module isfurther configurable to control operation of the first camera imagesensor and/or the first ISP in a fully operational mode and operation ofthe second camera image sensor and/or the second ISP in a partiallyoperational mode.

In an exemplary embodiment, the first camera includes a first cameraimage sensor that communicates with an associated first image signalprocessor (ISP) and is operative to output a first stream of frames, thesecond camera includes a second camera image sensor that communicateswith an associated second ISP and is operative to output a second streamof frames, and the synchronization and operation control module isfurther configurable to control operation of the first camera imagesensor and/or the first ISP in a partially operational mode andoperation of the second camera image sensor and/or the second ISP in afully operational mode.

In an exemplary embodiment, the synchronization and operation controlmodule is further configurable to synchronize pairs of frames processedby the first ISP and the second ISP.

In an exemplary embodiment, the synchronization and operation controlmodule is further configurable to synchronize pairs of frames processedby the first ISP and the second ISP.

In an exemplary embodiment, the control of the operation of the firstcamera image sensor in a fully operational mode and control of theoperation of the second camera image sensor in a partially operationalmode includes control of a respective frame size of each of the firstand second camera image sensors.

In an exemplary embodiment, the control of the operation of the firstcamera image sensor in a fully operational mode and control of theoperation of the second camera image sensor in a partially operationalmode includes control of a respective frame rate of each of the firstand second camera image sensors.

In an exemplary embodiment, the control of the operation of the firstcamera image sensor in a fully operational mode and control of theoperation of the second camera image sensor in a partially operationalmode includes control of a respective processing rate of each of thefirst and second ISPs.

In an exemplary embodiment, the system further comprises a smoothtransition library for providing to the synchronization and operationcontrol module an instruction used in configuring the synchronizationand operation control module to control operation of each camera and tooutput the dual-aperture camera output.

In an exemplary embodiment, the frame size of the camera in partiallyoperational mode is a fraction of the frame size of the camera in fullyoperational mode. In an exemplary embodiment, the frame rate of thecamera in partially operational mode is a fraction of the frame rate ofthe camera in fully operational mode. In an exemplary embodiment, theISP processing rate of the camera in partially operational mode is afraction of the ISP processing rate of the camera in fully operationalmode. Exemplarily, the value of the fraction may be a third. Thefraction may of course assume any another value smaller than 1. Forexample, the fraction may range between ¼ and ½.

In exemplary embodiments, there is provided a method comprisingproviding a dual-aperture camera that includes a first camera operativeto output a respective first camera output and a second camera operativeto output a respective second camera output, and operating one camera ina fully operational mode and operating the other camera in a partiallyoperational mode, thereby reducing dual-camera power consumption incomparison with a power consumption when both cameras operate in fullyoperational mode.

In an exemplary embodiment, the method further comprises outputting theoutput of the camera operating in fully operational mode as adual-aperture camera output.

In an exemplary embodiment, the method further comprises switchingbetween the first and second cameras and operating the second camera ina fully operational mode and the first camera in a partially operationalmode.

In an exemplary embodiment, the camera output includes a respectivestream of frames, and the method further comprises synchronizing aparameter of the output of the camera operating in fully operationalmode with a parameter of the output of the camera operating in partiallyoperational mode before outputting a dual-camera output.

BRIEF DESCRIPTION OF THE DRAWINGS

Aspects, embodiments and features disclosed herein will become apparentfrom the following detailed description when considered in conjunctionwith the accompanying drawings.

FIG. 1 shows schematically a system according to an exemplary embodimentdisclosed herein.

DETAILED DESCRIPTION

Embodiments disclosed below relate to dual-aperture cameras with reducedpower consumption and methods for operating such cameras. Each camera inthe dual-aperture camera includes a camera image sensor (or simply“sensor”) which is in communication with an associated image signalprocessor (ISP). In some embodiments, the two sensors may be associatedwith a single ISP and time-share it. The reduced power consumptionarises from the fact that most of the time one camera is “fullyoperational” while the other camera is less than fully operational or“partially operational”. As used herein, “fully operational camera” or“camera in fully operational mode” refers to a camera in which theoperation of the respective image sensor is at a regular frame rate orframe size and the operation of the associated ISP is at a regular ISPprocessing rate. As used herein, “partially operational camera” or“camera in partially operational mode” refers to a camera in which theoperation of the respective image sensor is at a reduced frame rate orframe size relative to its fully operational frame rate or frame size,and/or that the operation of the associated ISP is at a reducedprocessing rate relative to its regular (fully operational) ISPprocessing rate. In one example, the fully operational camera may outputframes at 30-60 frames per second (FPS), while the partially operationalcamera may output frames at a lower rate of 5-10 FPS. In anotherexample, the fully operational camera may output frames at 13 Mpxl sizeper frame, while the partially operational camera may output frames at alower size of 0.2-8 Mpxl per frame.

A synchronization mechanism enables fast switching of the dual-aperturecamera output from the output of the fully operational camera to theoutput of the partially operational camera. The synchronizationmechanism may be performed exemplarily by a dedicated software (SW)module. Alternatively, the synchronization mechanism may be included inhardware (HW). The novel synchronization mechanism and method presentedbelow allow synchronization of two camera streams even when one camerais fully operational while the other camera is partially operational.The disclosed synchronization keeps frames of the two camerassynchronized while reducing power consumption, and allows calculation ofa depth map from respective frames of the two cameras at all times.

Switching between cameras is decided by user inputs, such as selectionof zoom factor or scene, and is done by a smooth transition library, seebelow. Exemplarily, the library is a smooth transition library. The waya decision on the timing of transition is made is known in the art, seee.g. co-owned U.S. Pat. No. 9,185,291. Just before the dual-aperturecamera output is switched from the output of the fully operationalcamera to the output of the partially operational camera, the partiallyoperational camera becomes fully operational with respective fullyoperational frame rate and/or frame size and/or ISP processing rate.After the dual-aperture camera output switching, the fully operationalcamera becomes partially operational, with respective partiallyoperational frame rate and/or frame size and/or ISP processing rate. Ifthe fully operational camera was to stop streaming frames instead ofbeing switched to “partially operational” mode, the switching time fromthe fully operational camera to the partial operational camera wouldhave increased compared to the switching time when using the proposedoperation of the partially operational camera. If the partiallyoperational camera was to operate at full rate and a full frame sizewhile the fully operational camera outputs frames, the power consumptionof the entire dual-camera system would have increased, compared to thepower consumption when using the proposed operation of the partiallyoperational camera.

FIG. 1 shows schematically a system 100, according to an exemplaryembodiment disclosed herein. System 100 includes a dual-aperture camera110 with two camera image sensors (or simply “sensors”) 106 and 110.Camera 110 further includes several mechanisms (not shown in FIG. 1)such as camera body, lenses, actuation mechanism, light filters, etc. asknown in the art for camera module design (see for example patentapplications PCT/IB2014/062180, PCT/IB2014/063393 andPCT/IB2016/050844). Each sensor is associated with a respective cameraand its respective components. Exemplarily, system 100 can be adual-aperture zoom camera in which case sensor 106 is associated with aWide field of view (FOV) lens (not shown) and sensor 110 is associatedwith a Tele (narrow) FOV lens (not shown). In some embodiments, onesensor may be a color sensor (with a color filter array (CFA) such as aBayer array on the sensor pixels) and the other sensor may be amonochromatic sensor (without a CFA on its pixels). System 100 furtherincludes two image signal processors (ISPs), ISP 112 and ISP 114associated respectively with sensors 106 and 110. Each ISP processes asingle camera output frame and generates a processed frame. Processingsteps may include signal pedestal determination and removal, whitebalance, de-noising, removal of lens shading effect, de-mosaicing,sharpening, color correction, gamma correction, cropping and scaling ofthe frame and other steps, as known in the art. ISP 112 is connected tosensor 106 via one or more digital control channels 116 per camera (e.g.MIPI, I2C, SPI), and ISP 114 is connected to sensor 110 via one or moredigital control channels 118 per camera (e.g. MIPI, I2C, SPI).

System 100 further includes a synchronization and operation controlmodule 120 (in short and for simplicity “module 120”). Module 120controls the frame rates and/or sizes at which sensors 106 and 110operate and the processing rates at which the associated ISPs 112 and114 operate and is also in charge of synchronizing pairs of frames thatare processed and output by ISPs 112 and 114. System 100 furtherincludes a smooth transition library 130. Module 120 receives frames andframe parameters such as time stamps and requested frame rates from ISPs112 and 114 and/or from smooth transition library 130, and operatesbased on these parameters. Module 120 communicates with sensors 106 and110 through, respectively, digital control channels 122 and 124.

The frames output by ISPs 112 and 114 are passed to smooth transitionlibrary 130 along with other parameters such as frames sizes, exposuretime, analog gain information, ISP crop and scale information, framerate, focus position information and requested zoom factor. Smoothtransition library 130 is responsible for smoothly transitioning fromone stream of frames to another stream of frames, depending on severalparameters such as zoom factor, object depths in the pictured scene,etc. The smooth transition library can send a signal to module 120 tochange a partially operational camera to be a fully operational cameraand vice versa, and/or to change the frame size and/or to change theframe rate of the camera sensor and/or to change the processing rate ofthe respective ISP.

In some embodiments, the control of the frame rate by module 120 may beperformed via increasing or decreasing the vertical blanking time ofsensors 106 and 110. Increasing the vertical blanking time reduces theframe rate, while decreasing the vertical blanking time increases theframe rate.

Modules 120 and 130 may be software modules or may be implemented inhardware (HW). They may be included in a single HW processor or inseveral HW processors. Modules 112 and 114 are usually HW modules. ISP112 and ISP 114 may be implemented in separate HW modules (e.g.micro-processor, CPU, GPU, dedicated hardware, FPGA etc.) or in a singleHW module.

Following is a first embodiment of an exemplary method of operation ofsystem 100 and in particular of operation of module 120 in a desiredscenario in which one sensor (110) streams frames at a low frame rateand the other sensor (106) streams frames at a high frame rate. Thus inthis example, sensor 110 is the sensor of the “partially operational”camera and sensor 106 is the sensor of the “fully operational” camera.It should be noted that the role of low-frame-rate-sensor (partiallyoperational) and high-frame-rate-sensor (fully operational) isinterchangeable within the same system, and their definitions depend onparameters such as zoom factor and pictured scene information, and shownhere only by means of example. It should also be noted that in thisscenario, the operation rates of ISPs 112 and 114 match the rate of thestreams of frames that arrive from sensors 106 and 110, respectively.For example, if sensor 110 streams at low frame rate, the operation rateof ISP 114 is reduced compared to that of ISP 112, which receives framesat a higher frame rate from sensor 106. In the exemplary operation andin detail:

1. Module 120 configures the vertical blanking time of sensor 106 to avalue such that the sensor streams frames at a high frame rate, forexample 30 FPS. Module 120 also configures the vertical blanking time ofsensor 110 to a value such that it streams frames at a rate that is aninteger divisor (fraction) of the high frame rate of sensor 106 (e.g.such that the frame rate ratio between the rates is 1/n, n beingexemplarily an integer equal to or larger than 2).

2. Module 120 operates at the same frame rate as the high frame ratesensor. It continuously receives a new pair of frames from ISPs 112 and114, along with meta-data information such as frame time stamps for eachframe and a valid/invalid descriptor, which indicates whether the inputframes are valid ones or not. The frames streamed from sensor 106 areall marked as “valid” frames. Frames streamed from sensor 110 are alsomarked as “valid” frames. However, if there is a pair of frames in whichone frame from sensor 106 is valid and there is no corresponding framefrom sensor 110, then a “dummy” frame may be used instead of the missinglow-frame-rate frame and such dummy frame is marked as an “invalid”frame. For example, if sensor 106 streams at 30 FPS and sensor 110streams at ⅓ of the high frame rate (i.e. at 10 FPS), then module 120will receive a valid frame from sensor 106 about every 1/30 second and avalid frame from sensor about 110 about every 1/10 second. Since module120 operates at the high frame rate, it will receive two valid framesonly every 3^(rd) operation. Alternatively, module 120 may be calledonly when two valid input frames are available.

3. Module 120 compares the time stamps of the valid pair of frames andcalculates the time difference between them. It then calculates therequired modifications to the vertical blanking time of sensors 106and/or 110 so that the time difference between the two valid frames willbe minimized, and configures sensor 110 and/or sensor 106 to a newvertical blanking time.

4. Changes are applied to sensors 106 and/or 110 by sending a commandthrough digital control channels 122 and 124 (e.g. I2C channels).

5. The requested frame rate from each of sensors 106 and 110 can bedecided based on smooth transition library 130 requests. Smoothtransition library 130 may request control module 120 to configure thesame frame rate and/or frame size or different frame rates and/or framesizes from sensors 106 and 110, based on different parameters such aszoom factor, scene parameters, system performance and user preference.

Following is a second embodiment of an exemplary method of operation ofsystem 100 and in particular of module 120 in a desired scenario inwhich sensors 106 and 110 both stream frames at a high frame rate, ISP114 processes frames at a low frame rate and ISP 112 processes frames ata high frame rate. Thus in this example, ISP 114 is the ISP of“partially operational” camera and ISP 112 is the ISP of “fullyoperational” camera. In this scenario, module 120 only controls theprocessing rates of ISP 112 and ISP 114. Frames that reach ISP 114 athigh frame rate and are not processed by it are discarded. It should benoted that the role of low-frame-rate-sensor and high-frame-rate-sensoris interchangeable within the same system, and their definitions dependon parameters such as zoom factor and pictured scene information, andthe exemplary selected roles are shown here only by means of example. Inthe exemplary operation and in detail:

1. Module 120 configures both the vertical blanking times of sensor 106and sensor 110 and also the rates at which ISP 112 and ISP 114 operate.For example, sensors 106 and 110 are configured to stream frames at 30FPS, ISP 112 is configured to operate at a rate equivalent to 30 FPS andISP 114 is configured to operate at a rate equivalent to 10 FPS. Therate at which ISP 114 is configured to operate is set to be an integerdivisor of the rate that ISP 112 is configured to operate in (e.g. sothat the frame rate ratio between the rates is 1/n, n being an integerequal to or larger than 2).

2. Module 120 operates at the same frame rate as the high frame rate ISP112. It continuously receives a new pair of frames from ISPs 112 and114, along with meta-data information such as frame time stamps for eachframe and a valid/invalid descriptor, which indicates whether the inputframes are valid ones or not. The frames streamed from ISP 112 are allmarked as “valid” frames. Frames streamed from ISP 114 are also markedas “valid” frames. However, if there is a pair of frames in which oneframe arrives from ISP 112 and there is no corresponding frame from ISP114, then a “dummy” frame may be used instead of the missinglow-frame-rate frame and it is marked as an “invalid” frame. Forexample, if ISP 112 processes frames at 30 FPS and ISP 114 processesframes at ⅓ of the high frame rate, then module 120 will receive a validframe from ISP 112 about every 1/30 second and a valid frame from ISP114 about every 1/10 second. Since module 120 operates at the high framerate, then it will receive two valid frames only every 3^(rd) operation.Alternatively, module 120 may be called only when two valid input framesare available.

3. Module 120 compares the time stamps of the valid pair of frames andcalculates the time difference between them. It then calculates therequired modifications to the operation rates of ISP 112 and ISP 114 andalso the modification to the vertical blanking time of sensors 106and/or 110, such that the time difference between the two valid frameswill be minimized.

4. Changes are applied to sensors 106 and/or 110 via sending a commandthrough digital control channels 122 and 124 (e.g. I2C channels).

In both of the examples above, the requested frame rate from each ofsensors 106 and 110 and ISPs 112 and 114 can be decided based on smoothtransition library 130 requests as known in the art, see e.g. co-ownedU.S. Pat. No. 9,185,291. Library 130 may request module 120 to configurethe same frame rate or different frame rates from sensors 106 and 110and ISPs 112 and 114, based on different parameters like zoom factor,scene parameters, system performance and user preference.

Following is a third embodiment of an exemplary method of operation ofsystem 100 and in particular of module 120 in a desired scenario whereone sensor (110) streams frames at a low frame size and the other sensor(106) streams frames at a high frame size. In this case sensor 106 isthe sensor of the fully operational camera and sensor 110 is the sensorof the partially operational camera. It should be noted that the role oflow-frame-size-sensor (partially operational) and high-frame-size-sensor(fully operational) is interchangeable within the same system, and thattheir definitions depend on parameters such as zoom factor and picturedscene information, and shown here only by means of example. It shouldalso be noted that in this scenario, the ISPs 112 and 114 operationcomplexity depends on the frame streams that arrive from sensors 106 and110, respectively (for example, if sensor 110 streams at low frame size,the operation complexity of ISP 114 is reduced compared to that of ISP112, which receives frames at a high frame size from sensor 106):

1. Module 120 configures the vertical blanking time of both sensors 106and 110 to a value such that each sensor stream frames at a desiredframe rate, for example 30 FPS.

2. Module 120 also configures the frame size of sensor 106 to high framesize (e.g. 13 mpxl) and the frame size of sensor 110 to a low frame size(e.g. 0.5 mpxl). It further informs ISPs 112 and 114 on the expectedframe size for each ISP.

3. ISPs 112 and 114 set active and non-active hardware chains accordingto expected frame rate. ISPs 114 can for example reduce the number ofactive transistors (turn unneeded transistors off) and can reduce theoverall power consumption.

4. Module 120 operates at the same frame rate as the sensors. Itcontinuously receives a new pair of frames from ISPs 112 and 114, alongwith meta-data information such as frame time stamps for each frame.

5. Module 120 compares the time stamps of each pair of frames andcalculates the time difference between them. It then calculates therequired modifications to the vertical blanking times of sensors 106and/or 110 so that the time difference between the two valid frames willbe minimized, and configures sensor 110 and/or sensor 106 to a newvertical blanking time.

6. Changes are applied to sensors 106 and/or 110 via sending a commandthrough digital control channels 122 and 124 (e.g. I2C channels).

7. The requested frame size from each of sensors 106 and 110 can bedecided based on smooth transition library 130 requests. Smoothtransition library 130 may request SW synchronization and operationcontrol module 120 to configure the same frame size or different framesizes from sensors 106 and 110, based on different parameters like zoomfactor, scene parameters, system performance and user preference.

In a fourth embodiment of an exemplary method of operation of system100, system 100 may operate such that one sensor is streaming at fullframe rate and high frame size, while the second sensor operate at a lowframe rate and a low frame size. In this embodiment, there isimplementation of a combination of the operation methods presentedabove.

Table 1 shows a comparison of the four methods, with optional reductionof power. With the partially operational camera, each cell with “Full”text represents work at full power consumption, while each cell with“Partial” text represents reduction of power.

TABLE 1 Method 1 Method 2 Method 3 Method 4 Fully Sensor operation Full(e.g. 30FPS) operational rate (e.g. of 106) camera Frame size Full (e.g.13mpxl) ISP operation Full (e.g. 30FPS) rate (e.g. of 112) PartiallySensor operation Partial (e.g. Full (e.g. Full (e.g. Partial (e.g.operational rate 10FPS) 30FPS) 30FPS) 10FPS) camera (e.g. of 110) Framesize Full (e.g. Full (e.g. Partial (e.g. Partial (e.g. 13mpxl) 13mpxl)0.5mpxl) 0.5mpxl) ISP operation Partial (e.g. Partial (e.g. Full (e.g.Partial (e.g. rate (e.g. of 114) 10FPS) 10FPS) 30FPS) 10FPS)Synchronization and operation Full (e.g. 30FPS) control module 120operation rate Decision on transition\change in Done by smoothtransition library 130 frame rate

In summary, the present application discloses a system and methods foroperating the same, the system including a dual-camera in which thecombined preview or video output comes either from one camera or anothercamera, depending on user defined zoom factor, scene selection and otherparameters. Fast output switching with minimal power consumption penaltyis enabled by operating the camera not used to generate the dual-cameraimage output in a special (partial) operation mode.

The various features and steps discussed above, as well as other knownequivalents for each such feature or step, can be mixed and matched byone of ordinary skill in this art to perform methods in accordance withprinciples described herein. Although the disclosure has been providedin the context of certain embodiments and examples, it will beunderstood by those skilled in the art that the disclosure extendsbeyond the specifically described embodiments to other alternativeembodiments and/or uses and obvious modifications and equivalentsthereof. Accordingly, the disclosure is not intended to be limited bythe specific disclosures of embodiments herein. For example, while thisdescription is focused on a dual-aperture camera, multi-aperture cameraswith more than two apertures (cameras) may benefit from application ofthe methods described herein, if applied to any two cameras in amulti-aperture camera. In general, the disclosure is to be understood asnot limited by the specific embodiments described herein, but only bythe scope of the appended claims.

Unless otherwise stated, the use of the expression “and/or” between thelast two members of a list of options for selection indicates that aselection of one or more of the listed options is appropriate and may bemade.

It should be understood that where the claims or specification refer to“a” or “an” element, such reference is not to be construed as therebeing only one of that element.

All references mentioned in this specification are herein incorporatedin their entirety by reference into the specification, to the sameextent as if each individual reference was specifically and individuallyindicated to be incorporated herein by reference. In addition, citationor identification of any reference in this application shall not beconstrued as an admission that such reference is available as prior artto the present application.

What is claimed is:
 1. A method, comprising: providing a dual-aperturecamera that includes a first camera operative to output a respectivefirst camera output and a second camera operative to output a respectivesecond camera output; operating one camera in a fully operational modeto provide a first frame and operating the other camera in a partiallyoperational mode to provide a second frame; and obtaining a depth mapfrom the first and second frames, whereby the obtaining of the depth mapwith one camera in fully operational mode and the other camera inpartially operational mode requires less power consumption that thepower consumption required when both cameras operate in fullyoperational mode.
 2. The method of claim 1, further comprisingoutputting the depth map as a dual-camera output.
 3. The method of claim1, further comprising: switching between the first and second cameras;and operating the second camera in a fully operational mode and thefirst camera in a partially operational mode.
 4. The method of claim 1,wherein each camera output includes a respective stream of frames, themethod further comprising: synchronizing a parameter of the output ofthe camera operating in fully operational mode with a parameter of theoutput of the camera operating in partially operational mode before theobtaining of the depth map.
 5. The method of claim 4, wherein theparameter is a frame rate and wherein a frame rate of the cameraoperating in partially operational mode is a fraction of a frame rate ofthe operating in fully operational mode.
 6. The method of claim 4,wherein the parameter is a frame size and wherein a frame size of thecamera operating in partially operational mode is a fraction of a framesize of the operating in fully operational mode.
 7. The method of claim4, wherein each camera includes a respective image sensor and arespective image signal processor (ISP), wherein the parameter is an ISPprocessing rate and wherein a processing rate of the ISP of the cameraoperating in partially operational mode is a fraction of a processingrate of the ISP of camera operating in fully operational mode.
 8. Themethod of claim 2, further comprising: switching between the first andsecond cameras; and operating the second camera in a fully operationalmode and the first camera in a partially operational mode.
 9. The methodof claim 2, wherein each camera output includes a respective stream offrames, the method further comprising: synchronizing a parameter of theoutput of the camera operating in fully operational mode with aparameter of the output of the camera operating in partially operationalmode before the obtaining of the depth map.
 10. The method of claim 9,wherein the parameter is a frame rate and wherein a frame rate of thecamera operating in partially operational mode is a fraction of a framerate of the operating in fully operational mode.
 11. The method of claim9, wherein the parameter is a frame size and wherein a frame size of thecamera operating in partially operational mode is a fraction of a framesize of the operating in fully operational mode.
 12. The method of claim9, wherein each camera includes a respective image sensor and arespective image signal processor (ISP), wherein the parameter is an ISPprocessing rate and wherein a processing rate of the ISP of the cameraoperating in partially operational mode is a fraction of a processingrate of the ISP of camera operating in fully operational mode.
 13. Asystem, comprising: a dual-aperture camera that includes a first cameraoperative to output a respective first camera output and a second cameraoperative to output a respective second camera output; and asynchronization and operation control module configurable to controloperation of one camera in a fully operational mode to provide a firstframe and operation of the other camera in a partially operational modeto provide a second frame, and to obtain a depth map from the first andsecond frames, whereby the obtaining of the depth map with one camera infully operational mode and the other camera in partially operationalmode requires less power consumption that the power consumption requiredwhen both cameras operate in fully operational mode.
 14. The system ofclaim 13, wherein the synchronization and operation control module isfurther configurable to output the depth map as a dual-camera output.